fix(bedrock_guardrails): use Bedrock OUTPUT source for apply_guardrail when scanning model responses by shivamrawat1 · Pull Request #26144 · BerriAI/litellm

shivamrawat1 · 2026-04-21T02:44:16Z

Cause
BedrockGuardrail.apply_guardrail always called make_bedrock_api_request with source="INPUT", even when input_type="response" (post-call / model output). Bedrock guardrails often apply different policies for input vs output (e.g. PII/name rules only on output). Sending assistant text as INPUT led to action=NONE and no block.

Non-streaming completions go through unified_guardrail → OpenAIChatCompletionsHandler.process_output_response → apply_guardrail(..., input_type="response"), so they hit this bug. Streaming worked because that path already used source="OUTPUT" on the Bedrock call.

Fix
Map input_type to the Bedrock source: "request" → INPUT (messages), "response" → OUTPUT. For the OUTPUT path, build a synthetic ModelResponse whose choices carry the text to scan, and call make_bedrock_api_request(source="OUTPUT", response=synthetic_response, ...) so Bedrock evaluates output policies and blocks consistently with streaming.

… scans BedrockGuardrail.apply_guardrail hardcoded source="INPUT" regardless of the input_type parameter. On the non-streaming post-call path (unified_guardrail -> OpenAIChatCompletionsHandler.process_output_response -> apply_guardrail), the model response text was sent to Bedrock as INPUT, so guardrail policies configured for Output (e.g. PII/NAME blocking) returned action=NONE and the response passed through unblocked. The streaming path was unaffected because it calls make_bedrock_api_request(source="OUTPUT", ...) directly. Map input_type to the correct Bedrock source ("request" -> INPUT, "response" -> OUTPUT) and build a synthetic ModelResponse for the OUTPUT path so _create_bedrock_output_content_request produces the correct payload. Made-with: Cursor

greptile-apps · 2026-04-21T02:47:40Z

Greptile Summary

This PR fixes apply_guardrail in BedrockGuardrail to correctly pass source="OUTPUT" to the Bedrock ApplyGuardrail API when input_type="response", matching how the streaming path already behaved. It builds a synthetic ModelResponse with role="assistant" choices so make_bedrock_api_request can construct the right Bedrock OUTPUT payload, and adds two targeted mock-only regression tests covering both the INPUT and OUTPUT branches.

Confidence Score: 5/5

Safe to merge — the fix is narrowly scoped to the apply_guardrail OUTPUT path, is well-tested with mock-only regression tests, and addresses a real behavioural gap without touching other code paths.

All findings are P2 or lower; the previous P1 concern about test coverage for the OUTPUT branch was addressed by the author (two new tests added). The fix is correct, the logic is clear, and no regressions are introduced.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py	Fixes apply_guardrail to use source=OUTPUT when input_type="response", building a synthetic ModelResponse for the OUTPUT path; logic is correct and well-commented.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py	Adds two regression tests directly exercising the fixed INPUT/OUTPUT source routing; tests use mocks only (no network calls), asserting both the source param and synthetic ModelResponse shape.

Sequence Diagram

sequenceDiagram
    participant C as Caller (unified_guardrail)
    participant AG as apply_guardrail
    participant F as _prepare_guardrail_messages_for_role
    participant BR as make_bedrock_api_request

    C->>AG: apply_guardrail(inputs, input_type="response")
    AG->>AG: build mock_messages (role="user")
    AG->>F: _prepare_guardrail_messages_for_role(mock_messages)
    F-->>AG: filtered_messages
    AG->>AG: bedrock_source = "OUTPUT"
    AG->>AG: build synthetic ModelResponse (choices with role="assistant")
    AG->>BR: make_bedrock_api_request(source="OUTPUT", response=synthetic)
    BR-->>AG: bedrock_response
    AG-->>C: GenericGuardrailAPIInputs (masked/original texts)

    Note over C,AG: Before fix: always used source="INPUT" causing output-only policies to return action=NONE

_{Reviews (4): Last reviewed commit: "Merge remote-tracking branch 'upstream/l..." | Re-trigger Greptile}

…PUT source Add regression tests that mock make_bedrock_api_request and verify input_type=request uses source=INPUT with user messages, and input_type=response uses source=OUTPUT with synthetic ModelResponse. Made-with: Cursor

shivamrawat1 · 2026-04-21T02:54:45Z

@greptile review again with new commit

… litellm_post_call_non_streaming

gitguardian · 2026-04-21T18:46:21Z

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
29203053	Triggered	Generic Password	`62c2c55`	.circleci/config.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secret safely. Learn here the best practices.
Revoke and rotate this secret.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

… litellm_post_call_non_streaming

codecov · 2026-04-22T00:28:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

greptile-apps Bot reviewed Apr 21, 2026

View reviewed changes

Comment thread litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py

shivamrawat1 temporarily deployed to integration-postgres April 21, 2026 02:54 — with GitHub Actions Inactive

shivamrawat1 had a problem deploying to integration-postgres April 21, 2026 02:54 — with GitHub Actions Error

shivamrawat1 temporarily deployed to integration-postgres April 21, 2026 02:54 — with GitHub Actions Inactive

Merge remote-tracking branch 'upstream/litellm_internal_staging' into…

62c2c55

… litellm_post_call_non_streaming

shivamrawat1 temporarily deployed to integration-postgres April 21, 2026 18:45 — with GitHub Actions Inactive

shivamrawat1 had a problem deploying to integration-postgres April 21, 2026 18:45 — with GitHub Actions Error

Merge remote-tracking branch 'upstream/litellm_internal_staging' into…

e0cc158

… litellm_post_call_non_streaming

shivamrawat1 temporarily deployed to integration-postgres April 21, 2026 20:03 — with GitHub Actions Inactive

krrish-berri-2 approved these changes Apr 22, 2026

View reviewed changes

yuneng-berri merged commit d9494b6 into litellm_internal_staging Apr 22, 2026
101 checks passed

yuneng-berri deleted the litellm_post_call_non_streaming branch April 22, 2026 00:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(bedrock_guardrails): use Bedrock OUTPUT source for apply_guardrail when scanning model responses#26144

fix(bedrock_guardrails): use Bedrock OUTPUT source for apply_guardrail when scanning model responses#26144
yuneng-berri merged 4 commits intolitellm_internal_stagingfrom
litellm_post_call_non_streaming

shivamrawat1 commented Apr 21, 2026

Uh oh!

greptile-apps Bot commented Apr 21, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

shivamrawat1 commented Apr 21, 2026

Uh oh!

gitguardian Bot commented Apr 21, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shivamrawat1 commented Apr 21, 2026

Uh oh!

greptile-apps Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

shivamrawat1 commented Apr 21, 2026

Uh oh!

gitguardian Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Uh oh!

codecov Bot commented Apr 22, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

greptile-apps Bot commented Apr 21, 2026 •

edited

Loading

gitguardian Bot commented Apr 21, 2026 •

edited

Loading